Lecture 26 - Into Deep Learning P2

This is an R Markdown Notebook

Lecture 26 - Your project. Deep Learning with H2O. P.2 Build Deep Learning Model for our example

Deep Learning… This lecture is dedicated to the implementation of Deep Learning models into for our Data

Work overview (from previous lecture)

• re-arranging data as matrix
• fitting deep learning models
• testing the models
• saving models
• implementation in our ShinyApp…

The idea will be to use one particular feature e.g. Tubing Process, Impedance to play with. We will not yet implement this in Shiny but rather focus to prepare our data, fit and test the model

Re-arranging data as a matrix

Let’s get our time - series data as a dataframe first…

library(tidyverse)
library(plotly)
# ============= READ DATA =================
# Read our small data ... 
DF_Data_Recent <- readRDS("DF_Data_Process_Recent.data") 
DF_Equipm <- read_csv("DF_EquipmData.csv")
# data frame containing Event Names
DF_EvCode <- read_csv("DF_EvCodeDataProject.csv")
# Data manipulation and saving to the DF_TEMP
DF_TEMP <- DF_Data_Recent %>% 
  # join to decode equipment serial number
  inner_join(DF_Equipm, by = "IDEquipment") %>% 
  # join to decode Event Code meaning
  inner_join(DF_EvCode, by = "EventCode") %>% 
  # select only column needed
  select(StartDate, Name, AnalogVal, EventText)

Then I will plot my data for all 4 machines as just to remember how it looks like…

# creating human readable data and visualize them
DF_TEMP %>% 
  filter(EventText == "Tubing Process, resistance Ohm") %>% 
  ggplot(aes(x = StartDate, y = AnalogVal, col = Name)) + geom_point()+facet_grid(~Name)

Looking on the chart above I can see that machine #1 seems to be the best. Let’s assume that I take this for granted using my ‘domain’ knowledge. In case you just a consultant or just a Data Scientist, you may want to find and consult this ‘person’. For this reason, I will take that data from Machine #1 as a reference to build my Deep Learning Model.

Following chunk of code will extract this dataset with a filter() function.

# extracting only one machine
DF_M1 <- DF_TEMP %>% 
  filter(EventText == "Tubing Process, resistance Ohm") %>% 
  filter(Name == "Machine #1") %>% 
  select(StartDate, AnalogVal) %>% 
  arrange(StartDate) 
head(DF_M1)

Transposing the data

Now we need to transpose our data from long to wide structure and to fit it to matrix! For the moment, we will ‘forget’ the StartDate values and simply parse the values into matrix of dimension say 150 columns and 20 rows… Of course one need to recover some basic R skills for that :) if not use stackoverflow… (How to turn a vector into a matrix in R?)[https://stackoverflow.com/questions/14614946/how-to-turn-a-vector-into-a-matrix-in-r]

But in order to make things more clear for you (and for me) I will start simple. I will limit the output to 50 pieces of values to have more clear understanding on the process we do.

NOTE: Play with the code yourself to better understand every step!

DF_M1 <- DF_TEMP %>% 
  filter(EventText == "Tubing Process, resistance Ohm") %>% 
  filter(Name == "Machine #1") %>% 
  arrange(StartDate) %>% 
  select(AnalogVal) %>% 
  head(50) %>% 
  t() %>%  # this brings us a matrix
  matrix(nrow = 5, ncol = 10, byrow = TRUE) # transforming that into matrix size 5rows and 10columns
DF_M1

     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10]
[1,]   50   44   43   49   43   50   44   50   44    50
[2,]   44   51   45   51   43   50   44   50   44    50
[3,]   43   49   43   49   45   51   44   51   45    51
[4,]   44   43   49   43   49   43   49   43   52    45
[5,]   51   44   50   44   50   44   45   51   44    50

Notice to set byrow argument to TRUE. This way we will keep track on how our data are populated. Every row will contain about 150 observations of the Time-Series data. This will be equivalent of known time period.

Coming back to our data. Let’s complete this procedure for entire dataset. Let’s use the recipe to do a final transformation:

DF_M1 <- DF_TEMP %>% 
  filter(EventText == "Tubing Process, resistance Ohm") %>% 
  filter(Name == "Machine #1") %>% 
  arrange(StartDate) %>% 
  select(AnalogVal) %>% 
  head(3000) %>% 
  t() %>%  # this brings us a matrix
  matrix(nrow = 20, ncol = 150, byrow = TRUE) # transforming that into matrix size 20x150

Wonderful, let’s try to see! our new object as an image!!!

Explore the matrix as a surface!

Let’s use plotly 3D graph to explore what we have got!

plot_ly(z = DF_M1, type = "surface")

Values from Machine 1 used for Training

This should be good enough to fit our deep learning model.

Generalize data preparation as a function

As mr Bill Gates once said: “I would rather hire a lazy person as he/she will find the easiest way to solve a problem…” we must be lazy! For that I will already now think about writing a function that will ‘prepare’ my test datasets. Take a moment to study this:

# build function converting time series data to matrix
to_matrix <- function(x, filter_Event, filter_Machine, n_cols) {
  ### PURPOSE: Transform Time Series Column of the dataframe to the matrix
  #            with specified number of columns. Number of rows will be automatically
  #            found and remaining data points discarded
  # # Uncomment variable to debug function
  # filter_Event <- "Tubing Process, resistance Ohm"
  # filter_Machine <- "Machine #4"
  # x <- DF_TEMP
  # n_cols <- 150
  
  # get intermediate object and dimension
  Step1 <- x %>% 
    filter(EventText == filter_Event) %>% 
    filter(Name == filter_Machine) %>% 
    arrange(StartDate) %>% 
    select(AnalogVal)
  # find number of rows of data frame
  nrows <- Step1 %>% nrow()
  # find the number of row in a matrix (Whole Rows), the value will have decimals...
  WN <- nrows/n_cols
  ## extract the whole number uncomment for debug/test
  # WN <- 19.2
  # WN <- 19.8
  if((WN - round(WN)) < 0){WN <- round(WN) - 1} else {WN <- round(WN)}
  # find number of rows to extract data
  n <- n_cols * WN
  # extract relevant matrix
  Step2 <- Step1 %>% 
    head(n) %>% #only use whole number to avoid errors
    t() %>%  # this brings us a matrix
    matrix(nrow = WN, ncol = n_cols, byrow = TRUE) # transforming that into matrix size 20x150
  # return the result of the function
  return(Step2)
}

This function can definitely be easily reused for our ShinyApp. It can be used to generate dataset for any other machine event. Code snippets below will use this function to generate ‘Test’ datasets.

A. For machine 2:

DF_M2 <- to_matrix(DF_TEMP, 
                   filter_Event = "Tubing Process, resistance Ohm",
                   filter_Machine = "Machine #2", 
                   n_cols = 150)

B. For machine 3:

DF_M3 <- to_matrix(DF_TEMP, 
                   filter_Event = "Tubing Process, resistance Ohm",
                   filter_Machine = "Machine #3", 
                   n_cols = 150)

C. For machine 4:

DF_M4 <- to_matrix(DF_TEMP, 
                   filter_Event = "Tubing Process, resistance Ohm",
                   filter_Machine = "Machine #4", 
                   n_cols = 150)

And we can also visualize that to confront:

For Machine 2

plot_ly(z = DF_M2, type = "surface")

Values from Machine 2 used for Test

For Machine 3 - our candidate for anomaly detection!

plot_ly(z = DF_M3, type = "surface")

Values from Machine 3 used for Test

For Machine 4 - Another candidate for anomaly detection!

plot_ly(z = DF_M4, type = "surface")

Values from Machine 4 used for Test

Fitting the Deep Learning Model

Launch the deep learning machine again…

# to load the library
library(h2o)
# to initialize the 'machine'
localH2O = h2o.init()

H2O is not running yet, starting it now...

Note:  In case of errors look at the following log files:
    C:\Users\fxtrams\AppData\Local\Temp\RtmpwjnYF6/h2o_fxtrams_started_from_r.out
    C:\Users\fxtrams\AppData\Local\Temp\RtmpwjnYF6/h2o_fxtrams_started_from_r.err

java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

Starting H2O JVM and connecting: . Connection successful!

R is connected to the H2O cluster: 
    H2O cluster uptime:         2 seconds 104 milliseconds 
    H2O cluster version:        3.14.0.7 
    H2O cluster version age:    24 days  
    H2O cluster name:           H2O_started_from_R_fxtrams_pfu905 
    H2O cluster total nodes:    1 
    H2O cluster total memory:   1.77 GB 
    H2O cluster total cores:    4 
    H2O cluster allowed cores:  4 
    H2O cluster healthy:        TRUE 
    H2O Connection ip:          localhost 
    H2O Connection port:        54321 
    H2O Connection proxy:       NA 
    H2O Internal Security:      FALSE 
    H2O API Extensions:         Algos, AutoML, Core V3, Core V4 
    R Version:                  R version 3.2.5 (2016-04-14) 

Load datasets to H2O

Then we will download the datasets into h2o. Remember H2O is just operated from R but it’s a computer besides!

# Import train data into the H2O cluster
train_M1 <- as.h2o(x = DF_M1, destination_frame = "train_M1")

  |                                                                                                                      
  |                                                                                                                |   0%
  |                                                                                                                      
  |================================================================================================================| 100%

# Also import our test datasets for Machines 2 and 3...
test_M2  <- as.h2o(x = DF_M2, destination_frame = "test_M2")

  |                                                                                                                      
  |                                                                                                                |   0%
  |                                                                                                                      
  |================================================================================================================| 100%

test_M3  <- as.h2o(x = DF_M3, destination_frame = "test_M3")

  |                                                                                                                      
  |                                                                                                                |   0%
  |                                                                                                                      
  |================================================================================================================| 100%

test_M4  <- as.h2o(x = DF_M4, destination_frame = "test_M4")

  |                                                                                                                      
  |                                                                                                                |   0%
  |                                                                                                                      
  |================================================================================================================| 100%

Building Deep Learning model with autoencoder

Now, once we know how our data looks like we can start to do our Anomaly Model.

# Train deep autoencoder learning model on "normal" 
# training data, y ignored 
normality_model <- h2o.deeplearning(
 x = names(train_M1), 
 training_frame = train_M1, 
 activation = "Tanh", 
 autoencoder = TRUE, 
 hidden = c(50,20,50), 
 sparse = TRUE,
 l1 = 1e-4, 
 epochs = 100)

  |                                                                                                                      
  |                                                                                                                |   0%
  |                                                                                                                      
  |================================================================================================================| 100%

Calculate MSE from the train dataset

Let’s use this model on our training dataset…

# computer error of the model
h2o.anomaly(normality_model, train_M1) %>% as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")

Getting to know what AI see:

# visually see it
test_recon_M1 <- h2o.predict(normality_model, train_M1) %>% as.matrix()

  |                                                                                                                      
  |                                                                                                                |   0%
  |                                                                                                                      
  |================================================================================================================| 100%

plot_ly(z = test_recon_M1, type = "surface")

Detect Anomaly using MSE value

Now we will try to use test datasets from machine 2.

# Compute reconstruction error with the Anomaly 
# detection app (MSE between output and input layers)
h2o.anomaly(normality_model, test_M2) %>%  as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")

What a mess you would say - all observations are anomalous?? Yes! In fact this is correct. Indeed, sometimes AI lead to such a discoveries… In reality we observe that Machine #2 is completely different from Machine #1…

And for Machine 3

h2o.anomaly(normality_model, test_M3) %>%  as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")

Machine 3 has only anomalies at certain periods. There are general good performance while at approximately 10% of time we have a periods of anomalous behaviour…

Machine 4 You may try to practice with Machine 4!

h2o.anomaly(normality_model, test_M4) %>%  as.data.frame() %>% plot.ts(ylim = c(0, 10), type = "p")

Machine 4 shows anomalies only at periods 5 and 8. Then the behavious stabilizes…

Saving Deep Learning Model for the future use

To use our model in our ShinyApp we will save it…

if(!file.exists("www/tmp/normality_model.bin")){
h2o.saveModel(normality_model, "www/tmp/normality_model.bin")
h2o.download_pojo(normality_model, "www/tmp", get_jar = TRUE)
}

And let’s not forget to switch off our cluster!

h2o.shutdown(prompt= FALSE)

[1] TRUE

Conclusion

in this lecture we have performed:

• Time-Series data transformation from DataFrame to Matrix
• Trained Autoencoder model on ‘best’ performing machine
• Model was tested on a ‘worse’ performing machine
• Model was saved in several formats for the further use

Next step

our next step will be to repeat the procedure but on our machine data.

• re-arranging data as matrix
• fitting deep learning models
• testing the models
• saving models
• implementation in our ShinyApp…

used references

example from: (https://dzone.com/articles/anomaly-detection-with-deep-learning-in-r-with-h2o)[https://dzone.com/articles/anomaly-detection-with-deep-learning-in-r-with-h2o]

More reading: (https://dzone.com/articles/the-basics-of-deep-learning-how-to-apply-it-to-pre?fromrel=true)[https://dzone.com/articles/the-basics-of-deep-learning-how-to-apply-it-to-pre?fromrel=true]

And: (https://shiring.github.io/machine_learning/2017/05/01/fraud)[https://shiring.github.io/machine_learning/2017/05/01/fraud]

paper: (https://arxiv.org/abs/1701.01887)[https://arxiv.org/abs/1701.01887] In this lecture we would explore the ‘technology’ on the sample and try to do this in 10 min lecture!